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'1  Abstract 

r  '  I-  •  r  > 

]  ^ 

We  propose  a  column-eliminating  and  a  lower  bound  updating  technique  for  the 
simplex  method  for  linear  programming.  A  pricing  criterion  is  developed  for  checking 
whether  or  not  a  dual  hyperpl2me  corresponding  to  a  column  intersects  a  simplex  contain¬ 
ing  all  of  the  optimal  dual  feasible  solutions.  If  the  dual  hyperplane  has  no  intersection 
with  this  simplex,  we  can  eliminate  the  corresponding  column  from  the  constraints.  As 
the  simplex  method  iterates,  the  working  constraint  matrix  eventually  eliminates  all 
columns  except  those  that  are  in  at  least  one  optimal  b^lsis.  ^  >  . ,  '  ,  r  ■  >  ^  j 

K 
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*  The  author  is  very  grateful  for  many  discussions  with  Professor  George  Dantzig 
and  his  invaluable  suggestions  about  this  work. 


1.  Introduction 


Techniques  for  solving  linear  programming  (LP)  have  been  intensely  studied  for 
four  decades.  The  birth  of  linear  programming  is  usually  identified  with  the  concurrent 
development  of  the  simplex  method  in  1947  by  George  Dantzig.  It  says  much  for  the 
algorithm’s  originator  that  the  simplex  method  still  remains  the  major  algorithm  used 
in  optimization  systems,  although  recently  interior  methods  are  serious  competitors. 

Two  more  recent  approaches  to  solving  linear  programs  are  the  ellipsoid  method 
(Khachiyan  [7])  and  the  projective  algorithm  (Karmarkar  [6]).  Recently,  Todd  [S]  and  I 
[9]  found  that  the  ellipsoid  method  and  Karmarkar’s  algorithm  are  closely  related.  Both  of 
these  methods  generate  a  shrinking  dual  ellipsoid  containing  the  optimal  dual  solutions. 
Therefore,  some  columns  may  be  eliminated  from  the  constraints  if  their  corresponding 
dual  hyperplanes  have  no  intersection  to  the  containing  ellipsoid. 

When  I  talked  with  George  Dantzig  regarding  this  eliminating  issue,  he  predicted 
that  a  similar  result  could  be  obtained  for  the  simplex  method.  Therefore,  we  started  a 
series  of  research  meetings  along  this  direction.  The  result  is  this  paper  on  a  criterion  for 
eliminating  columns  in  each  iteration  of  the  simplex  method.  In  this  report,  we  introduce 
the  notion  of  a  simplex  S‘  that  contains  all  optimal  feasible  dual  solutions.  When  the 
primal  is  in  canonical  form  and  feasible,  the  simplex  in  the  dual  space  is  the  negative 
orthant  and  a  half  space  whose  boundary  hyperplane  is  parallel  to  the  dual  objective.  At 
the  next  iteration  f  +  1,  a  separating  dual  hyperplane  deletes  part  of  the  simplex  S*  and 
forms  a  new  containing  simplex,  Thus,  we  developed  a  strong  column-eliminating 

theorem:  if  a  dual  hyperplanc  corresponding  to  column  j  has  no  intersection  with  the 
containing  simplex  5‘,  then  column  j  in  the  primal  constraint  matrix  cannot  be  in  an 
optimal  basis  and.  therefore  can  be  eliminated  from  further  computation.  As  the  simplex 
method  iterates,  the  working  constraint  matrix  ultimately  reduces  to  only  those  columns 
that  are  in  at  least  one  optimal  basis. 

As  Goldfarb  and  Hao  [5]  pointed  out,  the  theorem  derived  in  this  paper  is  similar  to 
the  one  proposed  by  Cheng  [1,  2].  In  particular.  Corollary  1  in  our  paper  is  identical  to 
Theorem  1  on  identifying  the  permanent  nonbasic  variables  for  the  simplex  method  in 


Cheng  [1].  A  complete  literature  on  identifying  permanent  nonbasic  and  basic  variables 
can  be  found  in  Cheng  [2].  However,  all  existing  techniques  need  to  approximate  the 
optimal  objective  value,  and  we  develop  the  theoretical  basis  for  a  lower  bound  updating 
technique.  Most  existing  techniques  need  the  ^lssumption  of  nondegeneracy,  and  we  relax 
this  assumption  after  initially  cissuming  it.  Moreover,  the  geometrical  interpretation  of 
the  containing  simplex  in  our  theorem  is  new  and  this  interpretation  may  lead  to  a 
pricing  rule  for  selecting  the  incoming  column  as  the  one  that  most  reduces  the  volume 
of  the  containing  simplex  of  iteration  t  +  1  relative  to  that  of  iteration  t.  We  also  give 
a  simple  numerical  example  to  show  how  this  column-eliminating  scheme  performs  in 
practice. 

2.  The  Dual  Containing  Simplex 

We  assume  for  the  moment: 

Assumptions 

1.  an  initial  basic  feasible  solution  is  known; 

2.  the  minimal  objective  value  is  known  in  advzmce; 

3.  every  vertex  of  the  feasible  polytope  is  nondegenerate. 

The  last  two  assumptions  will  be  dropped  later.  Without  losing  generality,  we  can 
describe  the  primal  LP  problem  in  terms  of  the  ceinonical  form  at  each  iteration  t  of  the 
simplex  method. 

LP  Canonical  Form 

PLP  minimize  z  =  cx 

subject  to  X  G  A”  =  {z  G  il”  :  Ax  =  b  and  x  >  0} 

where  A  €  e  is  the  vector  of  all  one’s,  6  >  0,  and 

A  =  i  I,  D)  and  c  =  (  0,  cq)  ■ 

~T 

The  initial  basic  feasible  solution  is  x®  =  (6  ,0)^  and  the  initial  objective  value  is 
cx”  =  0.  We  also  denote  by  ^*  <  0  the  given  minimal  objective  value.  The  dual  of  the 
above  linear  program  PLP  is 


DLP 


maximize  yb 

subject  to  y  £Y  =  {y  £  R"*  :  yA  <  c} 

where  the  row  vector  y  6  R"*.  For  all  feasible  x  e  X  and  y  G  K  we  have  by  the  weak 
duality  theorem  (Dantzig  [3]) 

yb  <  z*  <  z  =  cx.  ( 1 ) 

The  dual  solution  corresponding  to  the  initial  basic  feasible  solution  is  y^  —  0.  If 
c  >  0,  then  y°A  <  c  is  feasible  and  =  cx®  =  0  and  the  solution  x®  is  optimal. 
Otherwise  y®  =  0  is  jin  infeasible  dual  solution  whose  dual  objective  function  shares  the 
same  value  as  the  primal  objective  value  cx®  =  0.  We  also  denote  by  Hj  the  hyperplane 

Hj  =  {ye  R”'  :  ya,  =  c^}, 

and  by  Hj  the  half  space 

HJ  =  {y  G  :  ya,  <  c,}, 

where  a,  is  the  jth  column  of  A.  If  y®  =  0  is  feasible  for  DLP,  then  both  x®  and 
y®  solve  problems  PLP  and  DLP.  Othewise,  we  have  at  least  one  hyperplane,  say  Hk, 
which  separates  0  from  Y .  Obviously,  the  number  of  such  separating  hyperplanes  are  the 
number  of  cjs  with  negative  sign. 

Let 

A  =  cx  -  z*  =  ~z\  (2) 

Then  for  all  dual  optimal  feasible  solutions  y*  of  DLP, 

y*A  <  c  and  y*b  >  —A. 

In  particular,  the  first  m  basic  columns  of  A  state  y*  <  0,  so  that 

y*  <  0  and  y*6  >  —  A 
define  the  simplex  5'  for  iteration  t: 

S'  =  {yG  H"*  ;y  <0,  y6  >  -A}. 

Therefore,  S'  is  a  simplex  containing  all  optimal  dual  feasible  solutions  y*; 


3.  The  Column-Eliminating  Theorem 

If  X*  and  y*  are  optimal  solutions  to  PLP  and  DLP  respectively,  then  complementary 
slackness  conditions  hold: 


(cj  -  y*aj)x*  =0  for  1  <  j  <  n.  (4) 

Hence,  Cj  —  y*aj  >  0  implies  that  x*  =  0,  i.e.,  the  jth  column  is  not  in  any  optimal  basis 
and  hence  c^m  be  eliminated  from  the  problem.  A  sufficient  condition  that  cj  —  y*aj  >  0 
for  all  dual  feasible  y*  is  that  it  be  true  for  all  y*  G  S‘,  since  S‘  contains  all  optimal  dual 
feaisible  solutions  as  well  as  other  y.  It  suffices  therefore  to  show  that  the  hyperplane  Hj 
has  no  points  in  common  with  S‘,  or  that  S*  is  contained  in  the  interior  of  H~ ■. 

S*  ClntiHj).  (5) 

However,  5'  only  has  m  -f-  1  vertices,  namely 

1/0  =  0,  ¥•  =  (-A/l,)U,  for  1  <  i  <  m,  (6) 


where  Ui  is  the  ith  unit  vector.  Therefore,  S‘  C  Int(H j  )  if  and  only  if  these  vertices  V* 
are  strictly  in  the  interior  of  the  half  space  H~ .  Thus,  we  have 

Theorem  1 
If 

cj-yaj>0  for  y  =  F’,  t  =  (0, 1, ...,  m),  (7) 

then  the  jth  column  of  A  is  not  in  any  optimal  b^lsis  for  PLP. 

Proof.  If  (7)  is  true,  then  Cj  —  yoj  >  0  for  all  y  =  ^  XiV'  =  S*  where  A,  >  0,  ^  Ai  =  1. 
Since  2J1  optimal  dual  feasible  solutions  y*  £  S\ 


Cj  —  y*aj  >  0  =>  X*  =  0. 


Hence,  the  proof  simply  follows  from  the  complementary  slackness  conditions  discussed 
above.  Q.E.D. 


Furthermore,  we  can  explicitly  calculate  the  minimum  of  those  m  +  1  values  m 
Theorem  1.  Letting  aij  be  the  ith  component  of  column  Oj,  we  derive 

Corollary  1  (nondegenerate  case) 

If 

cy  +  Amin(0,  min  (^)  )  >  0,  (8) 

l<i<m 

then  the  jth  column  of  A  is  not  in  any  optimal  basis  for  PLP. 

In  order  to  apply  Theorem  1,  the  values  Oij,  bi,  and  cy  must  be  available  from  the 
canonical  form.  In  the  revised  simplex  method  only  bi  and  the  inverse  of  the  basis  in 
either  explicit  or  factorized  form  axe  available  at  the  start  of  iteration  t.  It  is  necesseiry 
to  compute  Cj  but  it  is  too  costly  to  compute  a^y  for  every  column  j  except  for  the 
selected  incoming  column  j  =  s.  However,  at  the  end  of  the  iteration  t,  the  updated 
values  a,y,  for  the  outgoing  column  jr  of  iteration  t  are  readily  at  hand  for  iteration  <  +  1 
and  Theorem  1  may  be  applied  to  jr-  For  ex2imple,  if  (8)  holds  for  updated  a,jy.,  6,,  and 
cy, ,  then  column  jr  will  never  reenter  the  basis.  This  results  in  the  following  corollary. 
Corollary  2  (non degenerate  case) 

Let  the  sth  column  be  the  incoming  column  in  the  simplex  method  and  Or,  be  the 
pivot.  In  either  of  the  following  two  cases, 

1)  all  a,,,  except  Orj,  are  nonpositive; 


h.  A 


for  all  Oig  >0,  i  ^  r. 


the  outgoing  column  jr  can  be  eliminated. 


Proof.  Note  that 


updated  A  =  A  +  —  >  0; 

Or* 

_  — c, 

updated  c^y.  =  ^ —  >  0; 


updated  br  =  >  0; 


‘V 


updated  bi  =  6,  —  >0,  i  ^  r; 

Ora 

updated  Urj,  =  r~  >  0; 

Ora 

updated  a,j,  =  _^** ,  i  ^  r. 

Ora 

Substituting  the  above  updated  values  into  (8),  the  condition  for  eliminating  column  jr 


becomes 


^a  I  /  A  bfCf  •  ! r\  •  (  Oig/Ora  ^  l/Or*  >  _ 

—  +  (A  +  r^)mm(0,nnn(= - -  _  ),  -  )  >  0. 

Ora  Ora  bi-brOig/ara  br/Ora 


Obviously,  if  <  0  for  all  i  ^  r,  the  second  term  is  zero  and  the  first  term  is 
positive.  Hence  column  jr  can  be  dropped  in  ceise  1. 

In  case  2  of  Corollary  2, 

■  /'rv  •  /-  ~OialOra  ,  ^lOra  ^  ^ 

mm(0,  mm(= - =-3 — 7z~)»  -  )  =  i:^) - = - = — • 

bi  —  brOig/ara  br/Ora  Ora  mini^ro  >o(=^^ - =^) 

Thus,  multiplying  (9)  by  Or,, 

-  A  +  brCgf  Ora  ^  n 

-c, - = - =—  >  0, 

min.yr,a,.>o(^  - 


which  is  equivalent  to 


.by  br  .  A  +  brCg/Or 

nun  {- - IT—)  >  - z - 

i^r,a,‘j>0  Qyg  Ora  Cj 


bi  A  -  „  _  ■  , 

zr-  >  TT-T  10^  a,,  >0  i  ^r. 

Oia  CJ 


Q.E.D. 


One  should  note  that  if  two  rows  tie  for  pivot,  the  outgoing  column  can  not  be 
eliminated  since  degeneracy  occurs  at  iteration  t  +  1  (Dantzig  [4]).  We  now  remove  the 
nondegeneracy  assumption,  i.e.,  we  allow  that  6  >  0  instead  of  6  >  0.  In  this  case,  S* 
may  not  be  bounded.  However,  Theorem  1  is  still  valid  if  we  define 


=  if6i>0 

\  {-oo)Ui  if  bi  =  0, 


and  Corollary  1  can  be  modified  eis 
Corollary  3  (degenerate  or  nondegenerate  case) 

If  for  some  i,bi  =  0  and  Oij  <  0,  then  don’t  eliminate  column  j  ;  otherwise  if 

Cj  +  Amin(  0,  min  ('^)  )  >  0, 

l<»<m,6,>0  bi 

then  the  jth  column  of  A  is  not  in  any  optimal  basis  for  PLP. 

Similarly,  Corollary  2  can  be  modified  as 
Corollary  4  (degenerate  or  nondegenerate  case) 

Let  sth  column  be  the  incoming  column  in  the  simplex  method  and  Urs  be  the 
In  either  of  the  following  two  cases, 

1)  all  Uia,  except  an,  are  nonpositive; 

2) 

:^>max(T^,  ^)  for  all  Oj,  >  0,  i  ^  r. 

Off  |Cj|  Qra 

the  outgoing  column  jr  can  be  eliminated. 


4.  The  Lower  Bound  Generating  Technique 

Until  now  we  have  assumed  that  the  minimal  objective  value  z*  is  known  in  advance. 
Actually,  all  of  the  above  theorems  and  corollaries  are  still  valid  if  z*  of  (2)  is  replaced 
by  a  lower  bound  2:°  for  z* .  At  each  step  of  the  simplex  method,  we  can  update  2°  to  a 
possibly  higher  lower  bound  by  using  the  following  techniques. 

Theorem  2 


Proof.  If  A  is  finite,  say  A  =  — 6^,  then 

c  -  SkOk  =  c  -  SkUicA  >  0, 

i.e.,  SkUk  is  a  feasible  solution  for  DLP.  Therefore,  from  fl), 

—  A  =  SkUkb  <  c.r  =  2 

for  all  feasible  x  G  A'’.  Q.E.D. 

The  following  corollary  resembles  Theorem  2  by  looking  for  a  dual  feasible  solution 
that  has  the  form  Se  ^ . 

Corollary  5 

Let 

^  _  J  —(c^b)  mi\Xf,^f{{6  :  c  —  6c^ A  >  0},  if  {(5  :  c  —  6e^A  >  0}  ^  0; 

\  00.  otherwise  . 

Then  for  all  feasible  x  G  A, 

2  =  cx  >  —  A. 

Thus,  A  can  be  calculated  using  a  ratio  test  on  A  against  c. 
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r 


5.  A  Numerical  Example 


Suppose  that  the  simplex  tableau  is  given  as  follows  (Dantzig  [3]  pp.  97): 


Rows 

Col  1 

Col  2 

Col  3 

Col  4 

Col  5 

RHS 

Row  1 

-3/2 

7/8 

0 

-3/8 

1 

1/2 

Row  2 

1/2 

-3/8 

1 

-1/8 

0 

3/2 

Cost 

12 

-1 

0 

2 

0 

0 

Using  (10),  we  obtain  <5i  =  —4/7  and  82  =  —00.  Hence,  A  =  4/7  from  Theorem  2. 
Now  using  Corollary  1,  we  have 


Cl  +  A min(  0, ^min^ (^)  )  =  12  +  (4/7)(-3)  >  0 

and 

C4  +  Amin(  0,  min  (-=^)  )  =  2  +  (4/7)(— 3/4)  >  0. 
i<«<2  bi 

Therefore,  Columns  1  and  4  can  be  eliminated  from  the  computational  tableau. 

Moreover,  only  the  pivot  012  =  7/8  in  the  incoming  column  (Col  2)  is  positive.  Using 
Corollary  4,  the  outgoing  column  (Col  5)  can  be  eliminated  from  the  tableau  as  well. 
Thus,  we  have  only  two  columns  (Col  2  and  Col  3)  left,  which  form  the  optimal  basis. 

6.  Conclusion 

We  have  proposed  column-eliminating  and  lower  bound  updating  techniques  in  the 
simplex  method  for  linear  programming.  A  pricing  criterion  is  developed  on  checking 
the  intersection  between  dual  hyperplanes  and  the  dual  simplex  containing  all  of  the 
optimal  dual  feasible  solutions.  Under  this  criterion,  some  columns  may  be  identified 
early  as  the  optimal  nonbasic  columns;  therefore  they  can  be  eliminated  in  the  course 
of  the  simplex  method.  As  the  simplex  method  iterates,  the  working  constraint  matrix 
ultimately  reduces  to  only  those  columns  that  are  in  some  optimal  basis 
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Abstract 


We  propose  a  column  eliminating  and  a  lower  bound  updating  technique  for 
the  simplex  method  for  linear  programming.  A  pricing  criterion  Is  developed 
for  checking  whether  or  not  a  dual  hyperplane  corresponding  to  a  column 
Intersects  a  simplex  containing  all  of  the  optimal  dual  feasible  solutions. 
If  the  dual  hyperplane  has  no  Intersection  with  this  simplex,  we  can 
eliminate  the  corresponding  column  from  the  constraints.  As  the  simplex 
method  iterates,  the  working  constraint  matrix  eventually  eliminates  all 
columns  except  those  that  are  In  at  least  one  optimal  basis. 
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